Confouding, Effect Modification, and Mediation

Lecture 06

Author

Chris Slaughter

1 Overview

1.1 Scientific questions

  • Most often scientific questions are translated into comparing the distribution of some response variable across groups of interest

  • Groups are defined by the predictor of interest (POI)

    • Categorical predictors of interest: Treatment or control, knockout or

      wild type, ethnic group

    • Continuous predictors of interest: Age, BMI, cholesterol, blood pressure

  • Often we need to consider additional variables other than POI because …

    • We want to make comparisons in different strata

      • e.g if we stratify by gender, we may get different answers to our scientific question in men and women
  • Groups being compared differ in other ways

  • Confounding: A variable that is related to both the outcome and predictor of interest

  • Less variability in the response if we control for other variables

  • Precision: If we restrict to looking within certain strata, may get smaller $\sigma^2$

  • Covariates other than the Predictor of Interest are included in the model as\…

    • Effect modifiers

    • Confounders

    • Precision variables

    • Not necessarily mutually exclusive

2 Effect Modification

  • The association between the Response and the Predictor of Interest differs in strata defined by the effect modifier

  • Statistical term: “Interaction” between the effect modifier and the POI

2.1 Effect modification depends on the measure of effect that you choose

  • Choice of summary measure: mean, median, geometric mean, odds, hazard

  • Choice of comparisons across groups: differences, ratios

2.2 Examples of Effect Modification

2.2.1 Example 1: Is serum LDL by gender modified by smoking?

Mean Mean Median Median
Women Men Women Men
No Smoke 120 122 120 115
Smoke 133 122 133 124
Diff -13 0 -13 -9
Ratio 0.90 1 0.90 0.93
  • Effect modification for mean, not really for median

    • Holds for both difference or ratio

2.2.2 Example 2: Creatinine by stroke (modified by gender?)

Mean Mean Median Median
Women Men Women Men
No Stroke 0.72 1.08 0.7 1.1
Stroke 1.01 1.51 1.0 1.5
Diff -0.29 -0.43 -0.3 -0.4
Ratio 0.71 0.72 0.70 0.73
  • Effect modification for difference, not really for ratio

    • True for Mean or median

2.2.3 Example 3: Stroke by smoking (modified by gender?)

Proportion Proportion Odds Odds
Women Men Women Men
No Smoke 0.10 0.16 0.03 0.19
Smoke 0.16 0.26 0.19 0.35
Diff -0.06 -0.10 -0.16 -0.16
Ratio 0.62 0.62 0.16 0.54
  • Proportion: No effect modification for ratio, small amount for difference

  • Odds: No effect modification for difference, yes for ratio

2.2.4 Example 4: Stroke by smoking (modified by CVD?)

Proportion Proportion Odds Odds
None CVD None CVD
No Smoke 0.02 0.33 0.02 0.50
Smoke 0.04 0.50 0.04 1.00
Diff -0.02 -0.17 -0.02 -0.50
Ratio 0.50 0.67 0.50 0.50
  • Effect Modficiation?

    • Proportion: Yes for ratio, yes for difference

    • Odds: Yes for difference, no for ratio

2.2.5 Example 5: CHD by current smoking (modified by gender?)

Proportion Proportion Odds Odds
Women Men Women Men
No Smoke 0.18 0.26 0.22 0.36
Smoke 0.05 0.24 0.05 0.32
Diff 0.13 0.02 0.17 0.03
Ratio 3.60 1.08 4.17 1.11
  • Effect Modfication?

    • Proportion: Yes for ratio, yes for difference

    • Odds: Yes for difference, yes for ratio

2.2.6 Example 6: CHD by ever smoke (modified by gender?)

Proportion Proportion Odds Odds
Women Men Women Men
Never Smoke 0.16 0.25 0.19 0.33
Ever Smoke 0.16 0.26 0.19 0.35
Diff 0.00 -0.01 0.00 -0.02
Ratio 1.00 0.96 1.00 0.95
  • Effect Modfication?

    • Proportion: No for ratio, no for difference

    • Odds: No for difference, no for ratio

2.2.7 Summary comments on examples

  • If there is an effect, will see effect modification on at least one of the difference and ratio scale

  • If there is no effect (example 6), will see no effect modification on both difference and ratio scale

  • In real world, will usually see effect modification on both scales. The real question is the effect modification scientifically meaningful.

    • If we find there is important effect modification, science will go forward estimating effects separately

    • Models with interaction terms are useful for testing if effect modification is present (statistically)

  • Aside: Be careful when comparing two ratios

    • How close are two ratios?

      • 0.20 and 0.25 VERSUS 5.0 and 4.0?

      • 0.10 and 0.15 VERSUS 10.0 and 6.7?

    • Compare the ratio of ratios, not the difference

    • We might consider ratios to be more different when both ratios are \(>1\) than when both are \(<1\). But, that would be wrong.

2.3 Analysis of Effect Modification

  • When the scientific question involves effect modification

    • Conduct analysis within each stratum separately

    • If we want to estimate the degree of effect modification or test its existence, use a regression model including

      • Predictor of interest (main effect)

      • Effect modifying variable (main effect)

      • A covariate modeling the interaction (usually a product)

    2.3.1 Impact of ignoring effect modification

    • By design or mistake, we sometimes do not model effect modification

    • Might perform

      • Unadjusted analysis: POI only

      • Adjusted analysis: POI and third variable, but no interaction term

    • If effect modification exists, an unadjusted analysis will give different results according to the association between the POI and effect modifier in the sample

  • If the POI and the effect modifier are not associated

    • Unadjusted analysis tends toward an (approximate) weighted average of the stratum specific effects

      • With means, exactly a weighted average

      • With odds and hazards, an approximate weighted average (because they are non-linear functions of the mean)

  • If the POI and the effect modifier are associated in the sample

    • The “average” effect is confounded and thus unreliable (variables can be both effect modifiers and confounders)
  • If effect modification exists, an analysis adjusting only for the third variable (but no interaction) will tend toward a weighted average of the stratum specific effects

    • Hence, an association in one stratum and not the other will make an adjusted analysis look like an association (provide the sample size is large enough)

3 Confounding

3.1 Simpson’s Paradox

  • Confounding has its roots in Simpson’s Paradox

  • Given binary variables \(Y\) (response), \(X\) (POI), and \(Z\) (strata) it is possible to have …

\[\textrm{Pr}(Y=1 | X=1, Z=1) > \textrm{Pr}(Y=1 | X=0, Z=1)\] \[\textrm{Pr}(Y=1 | X=1, Z=0) > \textrm{Pr}(Y=1 | X=0, Z=0)\]

… but to also have …

\[\textrm{Pr}(Y=1 | X=1) < \textrm{Pr}(Y=1 | X=0)\]

3.1.1 Example: Probability of death (Y) at two hospitals (X) stratified by poor patient condition (Z)

  • Question: Which hospital do you want to be treated at?
    • Consider the results overall (averaging over Z) and conditional on Z below
Overall Died Survived Death Rate
Hospital A 16 784 2.0%
Hospital B 63 2037 3.0%
Good Condition Died Survived Death Rate
Hospital A 8 592 1.3%
Hospital B 6 594 1.0%
Poor Condition Died Survived Death Rate
Hospital A 8 192 4.0%
Hospital B 57 1443 3.8%
  • Ignoring condition, Hospital B has a higher death rather. However, within both poor and good condition, Hospital B has a lower death rate.

    • Poor condition is a confounder. Hospital B has more subjects with poor condition and subjects with poor condition have a higher death rate.

3.2 Definition of Confounding

  • The association between a predictor of interest and the response is confounded by a third variable if

    • The third variable is associated with the predictor of interest in the sample, AND

    • The third variable is associated with the response

      • Causally (in truth)

      • In groups that are homogeneous with respect to the predictor of interest

      • Not in the causal pathway of interest

  • We must consider our belief about the causal relationships among the measured variables

    • There is no statistical test for causality

    • Inference about causation comes only from the study design

    • BUT, consideration of the causal relationships helps us to decide which statistical questions to answer

  • Classic confounder

    • A clear case of confounding occurs when some third variable is a “cause” of both the POI and response

We generally adjust for such a confounder

3.2.1 Directed Acyclyic Graph

flowchart LR
  X[Predictor\nof interest] -- Causal? --> Y[Outcome]
  X[Predictor\nof interest] <-- Association --> Z[Confounder]
  Z[Confounder] -- Causal --> Y[Outcome]
  • Example: Ice cream (POI), murder rate (outcome), and temperature (confounder) in New York City during the summer
flowchart LR
  X[Ice Cream] -- Causal? --> Y[Murder Rate]
  X[Ice Cream] <-- Association --> Z[Air temperature]
  Z[Air temperature] -- Causal --> Y[Murder Rate]

3.2.2 Causal pathways

A variable in the causal pathway of interest
  • Not a confounder, so we would not adjust for such a variable

  • If we did adjust, we would lose ability to detect associations between the POI and the outcome

  • Example: Second hand smoke (POI), stunted growth (confounder), FEV1 (outcome)

    • Scientific question is about the impact of smoking on lung function

      • Stunted growth addresses lung anatomy, not lung function, which we don’t care about it
flowchart LR
  X[Second hand smoke] --> Y[FEV1]
  X[Second hand smoke] --> Z[Stuntend growth]
  Z[Stunted growth] --> Y[FEV1]
A variable in the causal pathway not of interest
  • However, we want to adjust for a variable in a causal pathway that is not of interest
    • Example: Work stress causing ulcers by hormonal effects versus alcoholism
    • Directed Acyclyic Graph
      • We can adjust for alcoholism to estimate the path through horomonal effects
      • Alternatively, we can adjust for hormonal effects to estimate the effect through alcoholism
flowchart LR
  X[Work stress] --> W[Hormonal Effects]
  X[Work Stress] --> Z[Alcholism]
  W[Hormonal Effects] --> Y[Ulcers]
  Z[Alcholism] --> Y[Ulcers]
Surrogate for response
  • Adjustment for a surrogate is a bad idea

  • As the name implies, surrogates are a substitute for the response variable

  • Directed Acyclyic Graph where forced vital capacity (FVC) is a surrogate for forced exp

flowchart LR
  X[Second hand smoke] --> Z[FVC]
  Z[FVC] --> Y[FEV1]
Many other (complicated) patterns possible

3.3 Diagnosing Confounding

  • Confounding typically produces a difference between unadjusted and adjusted analyses

  • This symptom is not proof of confounding

  • Such a difference can occur when there is no confounding

  • Symptom is more indicative of confounding when modeling means (linear regression) than when modeling odds (logistic regression) or hazards (Cox, proportional hazards regression)

  • Estimates of association from unadjusted analysis are markedly different from estimates of association from adjusted analysis

  • Association within each stratum is similar to each other, but different from the association in the combined data

  • In linear regression, differences between adjusted and unadjusted analyses are diagnostic of confounding

  • Precision variables tend to change standard errors, but not slope estimates

  • Effect modification would show differences between adjusted analysis and unadjusted analysis, but would also show different associations in the strata

  • More difficult to diagnosis confounding with non-linear functions of the mean

  • Common non-linear functions: Odds (odds ratios), hazards (hazard ratios)

  • May show the symptoms of confounding when confounding is not present

  • Adjusting for precision variables can appear to be confounding

  • In logistic and PH regression, difference between adjusted and unadjusted analyses are more difficult to judge

  • Comparison in more homogeneous groups (i.e. after adjustment for a precision variable) will drive slope estimates away from the null

  • Example: Suppose you have a sample where 50% of the subjects die

  • What is the variability?

  • We can reduce this variability by changing \(p\), the probability of death

  • Estimate \(p\) in different stratum. One stratum may have a higher \(p\), another a lower \(p\).

  • By making the estimate more precise, we have also impacted the mean

4 Precision Variables

4.1 Overview

  • Sometimes the scientific question to be answered is chosen based on which questions can be answered most precisely

  • In general, questions can be answered more precisely when the within group distribution is less variable

  • Comparing groups that are similar with respect to other important risk factors decreases variability

  • The precision variability is independent of the cause of the response

  • If we adjust for such a variable, we tend to gain precision

  • Directed Acyclyic Graph:

flowchart LR
  X[Predictor] --> Y[Outcome]
  Z[Precision] --> Y[Outcome]
  • Standard errors are the key to precision

  • Greater precision is achieved with smaller standard errors

  • Standard errors are decreased by either increasing \(V\) or decreasing $n$

  • Typically: \(se(\hat{\theta}) = \sqrt{\frac{V}{n}}\)

  • Width of CI: \(2 \times (\textrm{crit value}) \times se(\hat{\theta})\)

  • Test statistic: \(Z = \frac{\hat{\theta} - \theta_0}{se(\hat{\theta})}\)

  • Options for increasing precision

  • Increase sample size

  • Decrease $V$

  • (Decrease confidence level)

4.2 Adjusting for Precision Variables

4.2.1 Precision for Difference of Independent Means

Independent observations where group 1 has a different mean and variance than group 2

$ Y_{ij} \sim (\mu_j, \sigma_j^2), j = 1, 2; i = 1, \ldots, n_j$

$n = n_1 + n_2$; $r = n_1 / n_2$

$\theta = \mu_1 - \mu_2$,

$\hat{\theta} = \overline{Y}_1 - \overline{Y}_2$

$V = (r+1)(\frac{\sigma_1^2}{r} + \sigma_2^2)$

$se(\hat{\theta}) = \sqrt{\frac{V}{n}} = \sqrt{\frac{\sigma_1^2}{n_1} + \frac{\sigma_2^2}{n_2}}$

Might control for some variable in order to decrease the within group

variability

Restrict population sampled

Standardize ancillary treatments

Standardize measurement procedure

#### Precision for Linear Regression

Independent continuous outcome associated with covariate ($X$)

$\textrm{ind } Y_i | X_i ~ \sim(\beta_0 + \beta_1 X_i, \sigma^2_{Y|X}), i = 1, \ldots, n$

$\theta = \beta_1, \hat{\theta} = \hat{\beta_1}$ from LS regression

$V = \frac{\sigma^2_{Y|X}}{\textrm{Var}(X)}$

$se(\hat{\theta}) = \sqrt{\frac{\hat{\sigma}^2_{Y|X}}{n \hat{\textrm{Var}}(X)}}$

Adjusting for covariates ($W$) decreases the within group standard

deviation

$\textrm{Var}(Y | X)$ versus $\textrm{Var}(Y | X, W)$

Independent continuous outcome associated with covariate ($X$) and

precision variable ($W$)

$\textrm{ind } Y_i | X_i, W_i ~ \sim(\beta_0 + \beta_1 X_i + \beta_2 W_i, \sigma^2_{Y|X,W}), i = 1, \ldots, n$

$\theta = \beta_1, \hat{\theta} = \hat{\beta_1}$ from LS regression

$V = \frac{\sigma^2_{Y|X,W}}{\textrm{Var}(X)(1-r^2_{X,W})}$

$se(\hat{\theta}) = \sqrt{\frac{\hat{\sigma}^2_{Y|X}}{n \hat{\textrm{Var}}(X)(1-r^2_{X,W})}}$

$\sigma^2_{Y|X,W} = \sigma^2_{Y|X} - \beta_2^2 \textrm{Var}(W | X)$

#### Precision for Difference of Proportions

When analyzing proportions (means), the mean variance relationship is

critical

Precision is greatest when proportion is close to 0 or 1

Greater homogeneity of groups makes results more deterministic (this is

the goal, at least)

Independent binary outcomes

$\textrm{ind } Y_{ij} \sim B(1, p_j), i = 1, \ldots, n_j; j = 1, 2$

$n = n_1 + n_2; r = n_1 / n_2$

$\theta = p_1 - p_2$,

$\hat{\theta} = \hat{p}_1 - \hat{p_2} = \overline{Y}_1 - \overline{Y}_2$

$\sigma^2_j = p_j(1-p_j)$

$V = (r+1)(\frac{\sigma_1^2}{r} + \sigma_2^2)$

$se(\hat{\theta}) = \sqrt{\frac{V}{n}} = \sqrt{\frac{\sigma^2_1}{n_1} + \frac{\sigma^2_2}{n_2}}$

#### Precision for Odds

When analyzing odds (a nonlinear function of the mean), adjusting for

precision variables results in more extreme estimates

$\textrm{Odds} = \frac{p}{1-p}$

Odds using average of stratum specific $p$ is not the average of stratum

specific odds

Example: Stroke by smoking (in CVD strata)

No association between smoking and CVD in the sample: 10% smokers in

each group

CVD is not a confounder, but is clearly a precision variable

Note that the unadjusted odds ratio is attenuated toward the null

compared to the adjusted odds ratios

-------- ------- ------ ------ ------ ------ ------ ------- ------- -------

$N$ $p$ odds $N$ $p$ odds $N$ $p$ odds

Smoke 1000 0.04 0.04 100 0.50 1.00 1100 0.082 0.089

Nonsmk 10000 0.02 0.02 1000 0.33 0.50 11000 0.048 0.051

Ratio 2.00 2.00 1.75

-------- ------- ------ ------ ------ ------ ------ ------- ------- -------

Diagnosing Confounding

----------------------

### Adjustment for Covariates

We include predictors in an analysis for a number of reasons. In order

of importance\…

1. Scientific question

2. Predictor of Interest

3. Effect Modifiers

4. Adjust for confounding

5. Gain precision

Adjustment for covariates changes the question being answered by the

statistical analysis

Adjustments can be made to isolate associations that are of particular

interest

When consulting with a scientist, it is often difficult to decide

whether the interest in an additional covariate is due to confounding,

effect modification, or precision

The distinction is important because I tend to treat these variable

differently in the analysis

Often the scientific question dictates inclusion of particular

predictors

Predictor of interest: The scientific parameter of interest can be

modeled by multiple predictors (e.g. dummy variables, polynomials,

splines)

Effect Modifiers: The scientific question relates to the detection of

effect modification

Confounders: The scientific question may be state in terms of adjusting

for known (or suspected) confounders

### Confounder Detection

Unanticipated confounding

Some times we must explore our data to assess whether our results were

confounded by some variable

Goal is to assess the “independent effect” of the predictor of interest

on the outcome

Confounders

Variables (causally) predictive of the outcome, but not in the causal

pathway

Best method: Think about the scientific problem beforehand (perhaps draw

DAG)

Using data, often assessed in the control group

Variables associated with the predictor of interest in the sample

Note that statistical significance is not relevant because this tells us

about associations in the population

Detection of confounding ultimately must rely on our best knowledge

about the possible scientific mechanisms

Effect of confounding: A confounder can make the association between the

predictor of interest and the response variable look\…

Stronger than the true association

Weaker than the true association

The complete reverse of the true association (“qualitative confounding”)

Graphical Methods for Visualizing Effect Modification, Confounding, and Precision

---------------------------------------------------------------------------------

Conduct stratified analysis to distinguish between

Effect modifiers

Confounders

Precision variables

### Effect Modifiers

Estimates of treatment effect differ among strata

When analyzing difference of means of continuous data, stratified smooth

curves of the data are non-parallel

Graphical techniques difficult in other settings

![image](./effectmod/effmodplot.pdf)

### Confounders

Estimates of treatment effect the same across strata, AND

Confounder is causally associated with the response, AND

Confounder associated with the POI in the sample

When analyzing difference of means of continuous data

Stratified smooth curve of data are parallel

Distribution of POI differs across strata

Unadjusted and adjusted analyses give different estimates

![image](./effectmod/confoundplot.pdf)

### Precision Variables

Estimates of treatment effect the same across strata, AND

Variable is causally associated with the response, AND

Variable is not associated with the POI in the sample

When analyzing difference of means of continuous data

Stratified smooth curve of data are parallel

Distribution of POI same across strata

Unadjusted and adjusted analyses give similar estimates but with smaller

standard errors

![image](./effectmod/precisionplot.pdf)